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ABSTRACT 

The World Wide Web influences education and our lives in many ways. Nowadays, Web- 
based homework has been becoming widespread practice in physics courses and some other 
courses as well. Although are some disputes whether this is an encouraging or risky 
development for student learning, there is limited research assessing the pedagogical effect 
of changing the medium from written, hand-graded homework to online oriented, computer- 
graded homework. In this study, web-oriented homework system is developed to assess 
students' introductory physics course performance. Later on, these results are compared 
with paper-based (peer) homework performance for mid enrollment physics courses. One of 
two identical sections of introductory physics course students received paper- based, hand 
graded group homework while the other received the individual web-based homework. Then 
two groups' on conceptual and problem-solving performance measures are compared. No 
significant differences were found in students' Force Concept Inventory (FCI) test scores; 
however, average homework performance scores were significant that could be attributed to 
the homework method used in favor of paper- based peer homework group. 

Keywords: Physics assessment; web- based homework; paper- based homework; FCI. 

I NTRODUCTI ON 

The World Wide Web is affecting education as well as our lives in many ways. So far, the 
major use of the www in teaching has been for finding and distributing information, much 
like an online library. However, as information technology evolves, the Web has been 
increasingly used for more interactive applications such as online homework and online 
testing probably most rapidly growing educational uses of the I nternet. Self-assessment of 
web submitted homework and hand-graded homework of peer assessment refer to those 
activities of learners in which they judge and evaluate their own products of work and those 
of their peers with similar learning backgrounds. Both types of assessment emphasize 
students' active participation in the evaluation process. The two assessment methods are 
rapidly becoming more pervasive and dominant (Paris & Paris, 2001; Shepard, 2000). 

I n conventional classroom assessments, teachers usually play a major role. I n contrast, self- 
and peer- assessments require that students execute the assessment of their learning results 
and those of their counterparts. These processes have been demonstrated to produce 
beneficial effects in the cognitive, meta-cognitive and affective aspects of students' learning 
(Falchikov & Boud, 1989; Topping, 1998). 

Although, several dated review articles have summarized studies of the relationship between 
homework and academic performance and homework in general has been appreciated as an 
important course element (Cooper, 1989; Keith, 1982; Paschal, Weinstein, & Walberg, 1984). 
One review that examined 15 quantitative studies (Paschal, Weinstein, & Walberg, 1984) 
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concluded that there is a moderately large positive effect of homework on achievement, 
especially homework that teachers grade or write comments on it. Another review (Cooper, 
1989) examined 120 studies of the effect of homework on achievement. Twenty of the 
studies compared the achievement of students (in grades 1-12) who were given homework 
assignments with students who were not. Cooper also reviewed 50 studies that correlated 
achievement with the amount of time students reported spending on homework, and found 
that in 43 studies (or 86% ), a positive correlation was found indicating that students who 
spent more time on homework received better grades; the remaining 7 studies indicated the 
opposite. As in the previous homework/ no- homework comparison, the effect was 
nonexistent for elementary school students, and largest for high school students. A positive 
correlation between time spent on homework and grades was also reported in a study of 
high achieving high school students (Tymms & Fitzgibbon, 1992). 

Electronic homework as a course element has even more positive effects than written 
homework affirm by some researchers (Dufrense et al., 2002; Ogilve, 2000; Thoennessen and 
Harrison, 1996). Dufrense et al. (2002) for example compared the effect of the electronic 
homework and the written homework on student achievement as measured by exam 
performance. They found both that electronic homework led to higher overall exam 
performance. Nevertheless, the study conducted by Bonham et al. (2001, 2003) found that 
no significant differences in student performance were found that could be attributed to the 
homework method used. 

THE CONSTRAI NTS OF PAPER-BASED ASSESSMENT 

Using paper-based methods have some constraints. The possible constraints to paper-based 
assessments may be compensated by technology: 

The first constraint is that there is a relative problem of displaying multimedia works using 
pens and paper. However, in a computer- programmed environment, text, sound and graphics 
are just a few types of input that may be accessible in the web-based environment. 

Secondly, computers could make possible the effectiveness in recording and compiling the 
results of scoring/ commentary. In scoring or providing commentary of the works using pen 
and paper, limitations become obvious when multiple objects of assessment are dealt with 
(Billington, 1997; Hughes & Large, 1993). Further, as it is difficult to make real-time 
compilations and summaries of the scoring results, students may have difficulty in getting 
immediate feedback on their personal scoring results or in having, the general picture right 
after the scoring activity is completed. With offering database technology, when 
incorporated with an appropriate user interface, could provide instructors and students with 
efficient tools for recording. I n addition, coupled with appropriate computation functions, it 
can even do speedy calculations as well as make quick summaries and presentations of the 
records of assessment to provide users immediate feedback. 

Thirdly, web-technology could provide students with more opportunities of peer interaction 
beyond the constraints from time and locations. The essence of self- and peer- assessment 
lies in personal interactions. However, given the limited time of a class, even finishing the 
assessment activities itself poses a challenge; little time is left for discussion. The web-based 
environment is characterized by its accessibility at any time and its provision of synchronous 
and asynchronous methods of communication; therefore, it is possible to conduct activities 
either within the classroom or in after-class situations. 

Fourthly, it can be increased the diversity of teachers' implementation of self- and peer- 
assessment. A remarkable variety of forms of implementing self- or peer- assessment is 
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possible. I n setting the standards of scoring, for example, there are the options of letting the 
instructor set the standards (e.g., Magin & Helmore, 2001) or allowing students to do it 
themselves (e.g., Orsmond, Merry, & Reiling, 2000). 

METHODS OF WEB- BASED TESTI NG 

Testing is used for two essential purposes: to provide feedback and evaluation. Feedback 
refers to the response regarding a critical analysis of students' work. Evaluation refers to the 
grading and recording of students' work for assessing their understanding of material. These 
two purposes are not mutually exclusive. A testing instrument, whether it is a homework 
assignment, quiz, exam, or practice test, can satisfy both purposes to a varying degree. For 
instance, a practice test is primarily used to provide feedback to students for their self- 
evaluation. On the other extreme, an exam is primarily used for evaluation. Evaluation and 
feedback have different goals and thus have different implementation requirements. Because 
evaluation is primarily used to record student responses and assign grades, security concerns 
such as verifying a student's identity, protecting answer keys, limiting access according to a 
specific time or location, and preventing unauthorized sharing of information need to be 
considered. The second use, feedback, is used to respond to students' input by providing 
"correct/ incorrect" responses, hints, and solutions or by engaging the student in additional 
learning activities much like present computerized tutorials. 

POSITIVE CHARACTERISTICS OF WEB- BASED HOMEWORK ASSESSMENT SYSTEM 

The positive sides of automated web- based homework assessment system could be 
summarized as follows (Titus et al., 1998): 

Pedagogical approaches 

Using automated submission and scoring of assignments, instructors can give students more 
frequent assignments and more questions on each assignment than is possible with 
traditional methods, thus increasing the time that students spend studying material, 
answering questions, and solving problems. In fact, the computer can control the path 
through the assignment if desired, making the better prepared student's progress more 
efficient while choosing a more gradual approach for the less able student. Using Web-based 
assessment, similar questioning techniques can easily be administered to a large number of 
students. Even when assignments are more for self-evaluation and practice than for formal 
testing, records can be made of time on task and correlated with success in the course. 
Reports of this information to future students can make using the Web-based assignments 
more edible. 

Reducing administrative effort 

I ncreasing students' time on task and continually administering homework is made possible 
by automated grading. The server takes over the tedious grading of papers, which is often 
the obstacle to assigning more frequent homework or quizzes. It saves grading time for the 
instructor (or hired graders) and improves the quality of class time spent in problem solving 
recitations. Automated scoring also allows the instructor to perform item analysis to 
determine which questions are best predictors of student performance. I nstructors can then 
fit their assignments, whether homework, quizzes, or exams, to include questions that are 
best for probing student understanding. 

I nstantaneous feedback 

With computer-aided assessment, students can receive immediate feedback about their 
progress. Surveys given to students indicate that immediate feedback is one of the most 
appreciated aspects of web-based assessment. For this reason, it is valuable to deliver 
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assignments with immediate feedback and assignments with delayed feedback. The former 
type encourages students to consider why they missed certain questions and the latter 
ensures that students fully consider their answers before submitting. 

NEGATIVE CHARACTERISTICS OF WEB-BASED HOMEWORK ASSESSMENT SYSTEM 

Web-based assessment can be effectively used for providing unbiased evaluation and 
feedback on a frequent basis that is immediate, platform independent, multimedia-enhanced, 
and automated. However, new technology brings new concerns. The negative sides of 
automated web- based homework assessment system could be summarized as follows (Titus 
etal., 1998): 

Failure of observing students works 

When grading, it is useful for physics instructors to view students' work, check their 
diagrams, and follow their reasoning. Unfortunately, this is not currently possible on the 
web. As grading becomes automated, the instructor may lose students' some perception into 
the problem solving processes. This may be the largest restriction to Web- based assessment 
when used for evaluation. 

Less diversity of questions and grading methods 

Automating the grading process eliminates certain types of questioning and grading. Yet, 
there are new types of questions that can be delivered on-line that cannot be delivered on a 
piece of paper. 

Security issues 

If the web is used for evaluation, security issues inevitably arise. How can the instructor be 
assured that a student is doing his or her own work? I n the case of homework, the grade is 
not heavily weighted, students are usually expected to work together, and password 
verification may be enough to deter less- motivated cheaters. For exams or quizzes, security 
is a vital issue since these elements often count for a large part of a course grade. I n this 
case, there are sufficient ways of insuring security. One such way is to use randomization. 
Numbers within questions or the order of choices in multiple-choice questions can be 
randomized. 

Technical difficulties 

When using the web for assessment, instructors must realize that technical problems could 
occur. For instance, the server may be down or server software may need to be updated. 
I nstructors must plan to handle these difficulties, and it must be clear to students what they 
should do in the inevitable event of a technical problem. 

Typically, it is sufficient to extend the assignment deadline for individual students who have 
such trouble. Students also need access to the internet. At this time, it is unreasonable to 
assume that every student owns a computer and has internet access from home. Therefore, a 
school should have computer labs where students can access the internet on a regular basis. 

PURPOSE AND RESEARCH QUESTIONS 

The main aim of this study is to develop web oriented, computer-graded homework system 
to assess students' introductory general physics- 1 course homework performance and 
conceptual understanding and to compare these findings with the result of paper-based peer 
homework group. Meanwhile, the following research questions will be posed: 
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> Is there any statistically significant difference between assigned groups' FCI pre- 
and posttest scores? 

> Is there any statistically significant difference between assigned groups' 
homework performance scores? 

METHODOLOGY 

Participants 

The participants for this study had chosen a sample of convenience from Computer Education 
department (They have many experiences about online and computer-based assessment). I n 
an introductory physics course, students were taught by peer instruction. Web-based 
homework group consisted of 41 students and paper- based homework group consisted of 37 
students. 

Study design and Procedures 

A two-group pretest- posttest design was used in this study. This is a quasi-experimental 
design in which one group is subjected to a treatment and the other is subjected to a control 
group (Fraenkel & Wallen, 1996). Students were registered for the two different sections 
through standard course registration system and were unaware of the homework method 
until it was announced at the first week of physics- 1 class. The physics- 1 course has two 
main exams, one of which is mid-term and the other is final exam. The homework 
performance scores in both groups were added to include the 20 % of the final grade of the 
course. One (experimental group) is received their homework via online quiz system where it 
was graded by computer. The other (control group) wrote out solutions to homework 
exercises on paper with working as a group consisted of four or five students. These 
exercises were turned in and graded by the instructor. Throughout the semester after each 
unit of physics- 1 course was completed; students were administered homework questions 
according to their assigned groups. Most of the homework exercises on which the two 
groups worked was the same or similar problems from the physics textbook (Turkish 
translation of Principals of Physics by Bueche and Jerde, sixth edition, 1995) with addition of 
some conceptual questions by the instructor. There were total eight homework exercises, in 
both groups, they were graded in percentage scores, and average scores calculated to be 
used as their homework performance scores. The Force Concept Inventory test was 
administered before (in first week of the semester) and after the instruction (the last second 
week of the semester) in both groups. FCI test scores (in percent average scores) and 
homework performance scores were then taken to SPSS packet programme to make 
necessary analyses. 

Web- based and paper- based peer homework groups 

Paper- based peer homework group consisted of four or five students each group and there 
were 37 students in total. The types of problems used in group homework assignments are 
nearly identical to those used in individual web-based homework assignments; in fact, the 
vast majority of the problems in the web based homework library come from the chapter 
problems of standard first year university algebra- based physics textbooks with addition to 
some conceptual questions. After finishing every unit -there were nine units-, students were 
given one assignment of five to nine problems. 

Gibson (2001) et al. suggested that the online testing should be one component of the 
evaluation of the student; therefore, in both systems 20% of the course final grade comes 
from the average homework score. The web-based homework system is called "online 
testing" and detailed description of the program is explained in the following section. 
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The structure of the Web- based homework system 

The Web-based homework system was developed by Linux-based php extension html 
environment with using MySQL database system and has two main modules. The first module 
is for students and the second is for instructor. 

The system's main goals are: 

> To provide instruction with a simple and easy online interface to create and 
manage online multiple choice and subjective exams, with such features as 
automatic grading, and feedback to students, 

> To let instructors and students check grades. 

In students' module, as shown in Figs, la, b and 2, first, students have to register to 
password protected web- based homework system. Once they enter the system whenever 
their homework are activated to ready for take it, they colud only see and able to take their 
test. After finishing the test, they could see their results from the test to get immediately 
feedback from it. Students may ask any question related to test or any problem that 
encountered any time just clicking provided link to communicate their instructor (Figure 3) 
to write e-mail messages. Then also, students could be able to see any time their homework 
results by clicking related link (Figure 4). 




Figure: la and lb 

Students' registration and logging in online homework system pages, respectively. 
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Figure: 2 

Ready to take some "online quizzes" by students 




Figure: 3 and 4 

Students' communication to the instructor and students' sample 
homework result 1 page, respectively. 

The instructor management module provides instructor with a convenient user interface that 
allows them to execute various setup and management functions online, such as setting up 
accounts, setting up test parameters, queries as to students' scoring process and observing 
various assessment results at any time. Web-based homework system offers parameters for 
instructor to configure the options of various types of activities (Sample activities are shown 
Figs. 5 to 10). 
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Figure: 7 

Defining number of homework questions for any quiz 
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Figure: 8 

Entering homework questions according to its unit and defining 
correct answer and level of difficulty. 
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69 soruluk BOLUM 8 - ( donme hareketi ve eneiji-momentum) (donme hareketi ile eneiji ve momentum korunumuna ait sorulan iyerir) 
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Figure: 9 and 10 

Activating/ Deactivating quizzes, and students' results pages, respectively. 
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By the instructor module, the instructor might do the following main tasks: 

> Create or delete homework assignment quizzes and define number of 
questions to be asked for each assignment. 

> To see students' homework results and progress, and see their detail of 
assignment results such as starting and finishing date and time, etc. 

> Answer students' e-mail messages to communicate with them to solve 
whatever problems they might have encountered during the process. 

> Able to activate or deactivate any particular assignment to students. 

> Define number of questions for each assignment. 

> Define the level of difficulties of those questions which uploaded the server 
(specifically in cognitive knowledge and conceptual questions labeled as 
"easy”, questions in application level labeled as "normal"; questions in 
complex application and analysis level labeled as "difficult", see figure 8). 

RESULTS AND Dl SCUSSI ONS 

This part consists of two sections, descriptive statistics, and inferential statistics. 

Descriptive Statistics 

The results of homework performance scores (in percentage), FCI pre-, and posttest scores, 

and normalized-gain scores for both groups are given in Table: 1. 

Table: 1 

The results of both groups' FCI tests and homework performance (percentage) scores 



Web- based homework-group Paper- based homework group 





N 


Average 


S. Deviation 


N 


Average 


S. Deviation 


Homework 

performance 


41 


71.15 


15.428 


37 


80.30 


7.237 


FCI pre-test 


40 


41.05 


11.89 


33 


42.73 


9.69 


FCI post-test 
FCI normalized gain 


39 


62.87 
% 37.01 


9.96 


36 


61.44 
% 32.66 


9.97 



It can be seen in Table 1 that average FCI pretest score (in percentage) in both 
groups are about in the forties and posttest scores are about in sixties, however homework 
performance percent score for web-based group are about 71%, while for paper-based 
group are about 80 % . Moreover, web- based homework group's FCI normalized gain score is 
about 37 % , the paper-based homework group's gain score found about 32.6 % . 

I nferential Statistics 

Both homework assignment groups' summary t-test results related to FCI pre- and post-test 
scores and homework performance scores are given in Table: 2. 



Table: 2 

T-test su mmar y results 



Test differences between groups 


df 


t-test 


P 


Homework performance 

differences 


76 


-3.29 


0.002* 


FCI pre-test differences 


71 


-0.65 


0.517 


FCI post-test differences 


73 


0.61 


0.539 



*p<0.05 
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It can be seen in Table 2 that there were not any statistical differences in FCI pre- and 
posttest scores in terms of assigned groups of being web-based or paper-based (t 7 i=-0.65, 
p>0.05 for FCI pretest; t 73 =0.61, p>0.05 for FCI posttest). However, statistical difference is 
found in favor of paper-based homework group in average homework performance scores 
with respect to assigned two groups (t 76 =-3.29, p<0.05). Average normalized gain 
percentage score of web- based group is higher than paper- based group's score, on the 
contrary, paper- based assignment group's average homework performance percent score is 
higher than those of web- based group are. This result is also not statistically significant at 
the 0.05 level as well. It is remarkable that although computers or web-based homework 
system may have advantages of supporting conceptual understanding (with higher FCI gain 
scores), these advantages could not continue in their homework performance score. 
Moreover, significant differences found in this study between web- based and paper- based 
homework groups shows that solving homework problems with peer groups could be 
effective than web- based homework system. 

CONCLUSI ON AND I MPLI CATI ONS 

Was the web- based testing effective on students' physics homework performance score and 
conceptual test (FCI) achievement score when compared to paper- based homework? There 
was a not statistically significant difference between web- based homework system and 
paper- based group homework system with respect to conceptual FCI pre- and posttest 
scores. However, Ogilve (2001) concluded that electronic homework tutoring led in 
producing gains in FCI score that were twice as large as those from the written problem set 
were. While comparison of their performances on regular exams, conceptual exams, quizzes, 
laboratory, and homework showed no significant differences between groups; other 
measures were found to be strong predictors of performance( Bonham at al., 2003), however, 
in this study, there was a statistically significant result between web- based and paper- based 
homework assignment results in favor of paper-based group. Dufresne, Mestre, Hart, and 
Rath, (2002) compared student performance over several years in large introductory physics 
courses, including both calculus- based and algebra-based courses, and four different 
instructors who had taught courses with both paper-based and web-based homework 
system. Student exam scores generally improved at a significant level after the introduction 
of web- based homework. Students using web- based homework reported spending 
significantly more time on assignments than did those using paper homework. The paper- 
based homework groups' students were required to work out the entire solution and show 
their work, whereas the Web- based homework groups' students needed only to submit the 
final numerical answer. 

The current literature does not really answer questions raised if computerized homework or 
web- based is more effective or otherwise around. Homework is important for technical 
courses such as introductory physics, where problem solving is a major focus and homework 
is the main venue for practicing. Although, Maloney (1994) claims that many students 
struggle to develop problem-solving skills in physics, it has been shown that directed 
instruction and feedback are effective in developing problem solving skills (Heller & Reif, 
1984; Heller & Hollabaugh, 1992). 

It has been argued that learning environments are more effective for collaborative learning, 
and in this study collaboration of paper-based group was found to be a significant factor 
related to average homework performance scores. This could be because of the presence of 
certain behavioral factors that can have significant effects on relationships between 
performances. For future study, it would be fruitful to examine such behaviors in relation to 
other factors associated with learning, such as learning styles, attitudes, environmental 
variables, and learning strategies and methods. Additional contextual factors should be 
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considered, to include students' extra-curricular activities. Such inquiries could prove to be 
some beneficial results (Kotas and Finck, 2002). 

Web- based homework is a viable alternative to the traditional paper- based approach. It does 
not bring significantly greater benefits to students, but neither does it do much worse than 
standard methods of collecting and grading homework. This supports the viewpoint that 
technology itself does not improve or harm student learning, but somewhat the underlying 
pedagogy is the vital issue. Automated homework system probably will help students in 
courses where homework could not otherwise be assigned. Students generally respond 
positively to using a computer for homework. 

Web-based homework may also allow for more pedagogically sound instruction by freeing up 
instructor resources for other aspects of the course, or by enabling new kinds of assignments 
that may be more valuable than traditional paper- based ones. Technology alone is not going 
to improve instruction, but web-based homework has a fair place in the physics instructor's 
toolbox. 

Self- and peer- assessments are methods of evaluation highly recommended by researchers, 
but the traditional pen-and- paper- based self- and peer- assessment method has many 
restrictions (Sung et al., 2005). Replacement of hand-graded homework by computer work 
could improve student learning by freeing time and economic resources for more effective 
instructional methods, and could be a medium that allows widespread use of exercises with 
greater pedagogical value (Bonham et al., 2003). 
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